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The subject matter of these Drive-In Conferences 
is selected after inquiry of our community college 
presidents and, through them, of their respective 
faculty members. All indicate that high on the 
priority list is the topic of institutional evaluation 
as applied to administration. 

In corporations, businesses, and governmental 
organizations across America, institutional evalu- 
ation has progressed to a highly specialized 
science. In the area of higher education, we have 
been somewhat behind the times in taking a hard 
and profitable look at the internal structures and 
operations within our own bailiwicks. But we have 
also been making impressive strides. 

In our own community college area we have 
access to several organizations designed in part 
for developing internal research at the community 
college and multiversity levels. Outstanding among 
these are the Educational Testing Service, the 
Center for the Study of Higher Education, Univer- 
sity of Michigan, and the Educational Development 
Program, Michigan State University. Each organi- 
zation is ably represented in this conference. 

The following reports are indicative Ol the services, 
techniques, and data of which we may avail our- 
selves and apply to our own administrative needs. 
Further inquiry is welcomed. 
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A Catalytic Agent for 
Innovation in 
Higher Education 

John E. Dietrich and F. Craig Johnson* 

Late in 1965 , John Gardner warned that ^*it will be 
possible for colleges and universities to be busy and 
populous and yet fail in their essential jobs —— which is 
to say that they could be busy and populous frauds/^ ^ 
To avoid this disasterous possibility, he challenges uni- 
versity educators to **restore the status of teaching, 

* 'undertake a thorough reform of the undergraduate 
curriculum,'' and '‘'improve our procedures for institu- 
tional planning." It is within the context of just such 
challenges that Michigan State University has developed 
a catalytic agent for change, the Educational Development 
Program (EDP). This program is helping in the forward 
planning of the institution and places particular emphasis 
on the improvement of undergraduate education. While 
programs of this type are certain to vary in form and 
scope from one institution to another, the basic concepts 
should be applicable to institutions of varying size and 
nature. ^ 

The Need for Innovation in Higher Education 



Again and again in the last few years, the need for innova- 
tion has been expressed, almost always in the most 
generic terms, such as limited resources, increased 
enrollments, shortage of faculty, the explosion of knowl- 
edge and the increasing demands of society for research. 
These forces are having a particularly direct and fre- 



*Mr. Dietrich is assistant provost and director of the Educational Development 
Program at Michigan State University. Mr. Johnson is assistant director of the 
Educational Development Program and assistant director of Institutional 
Research at the same university. 

^John W. Gardner, “Agenda for the Oalleges and Universities - Higher Ed- 
ucation in an Innovative Society,** Journal of Higher Education, Vol. XXXVI, 
No. 7, October, 1965, p. 359. 

^One author is a member of the board of trustees of a small private liberal 
arts college (600 students and 200 courses) which seems to be a microcosm 
of Michigan State (31,000 undergraduates and 2,250 undergraduate courses). 
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quently deleterious effect upon our undergraduate pro- 
grams, Let us look at . a few examples of the specific 
problems and questions which lie beneath the general 
descriptions. 

Limited Resources — Most new dollars are being used to 
cover the costs of new students. Thus the per capita 
dollars for the undergraduate are not rising significantly, 3 
Other new dollars are being employed to strengthen the 
costly graduate and research programs needed to meet 
the increased demands of society. As a result, the under- 
graduate program must be improved with approximately 
the same dollars in the face of a more expensive economy^ 
Where should any new dollars go? To increase present 
faculty salaries? To hire more faculty? To hire more 
teaching assistants? To hire technical supporting per- 
sonnel? To add new programs? To experiment with new 
technologies? To finance more efficient teaching models? 

Increased Enrollments — New enrollments coupled with 
the limited resources lead to another complex of problems. 
To suggest only a single aspect, the beginning service' ' 
courses (taking care of other people's kids — the non- 
majors) are becoming educational complexes ranging 
from hundreds to thousands of students in a single term. 
Recognizing that most faculties are striving to keep up with 
their disciplines and are not much interested in ''other 
people's kids," what can we do? Build more large lecture 
halls? Employ television? Develop independent study ma- 
terials. Equip programmed laboratories? Buy teaching 
machines? Computers? 

Faculty Shortages — While a shortage of faculty may be 
only a five to ten year interim problem, it is upon us. At 
present, there is a consistent move to turn more and more 
of undergraduate education, particularly the highly critical 
first two years, over to less qualified personnel. Some 
institutions admit to teaching 30 per cent or more of their 
undergraduate program with teaching assistants. This 
pattern is with us to stay. How can v/e improve our under- 
graduate teaching? Develop new patterns of orientation for 
new teachers? Give them supervision? In-service training? 
All of which requires senior faculty time. Or should we 
eliminate or reduce the number of small-group instruc- 
tional models? Develop "peer group" teaching? Team 
teaching? Build large course complexes run by mana- 
gerial personnel? 

Explosion of Knowledge — The growth of knowledge is so 
tremendous that we can no longer teach coverage. The 
questions are clear. What shall we teach? and How shall 
we teach it? Or perhaps better stated. What should be 
learned? and How shall the student learn it? How shall we 
organize knowledge ? How can we determine students' real 
needs? How can knowledge be made relevant to the stu- 
dent? Should we reorganize the curriculum? Reorganize 
course content? Experiment with new teaching methods? 
Develop use of new learning theories and applications? — 
All of which will compete for time and dollars. 

New Societal Demands — Intensified new research de- 
mands coupled with new national and international re- 
sponsibilities have thrust imposing new requirements on 
our already overextended faculties and facilities. Indeed, 
Jacques Barzun believes that "yielding to the claims of 
society will fragment and ultimately destroy an institu- 
tion." While we disagree, we must admit that each new 
demand from society does place a greater stress on 



^At Michigan State the numbers of dollars and students have increased tre- 
mendously, but the dollar per student has actually decreased slightly in the 
last five years. 



undergraduate education. How can we fulfill our traditional 
responsioility for educating? How can we reverse or re- 
tard the "flight from teaching?" How can we evaluate and 
reward teaching? How can we retrain our tenured facul- 
ties? And so on. These sample problems and questions 
should illustrate the interlocking nature of the difficulties 
with which our delicately-balanced, inordinately- con- 
servative, tradition-bound institutions of higher education 
must cope. 

Developing the Climate for Innovation 

Machiavelli once said, "It must be considered that there 
is nothing more difficult to carry out nor more doubtful 
of success nor more dangerous to ha .idle than to initiate 
a new order of things, for the reformer has enemies in 
all of those who profit by the old order and only lukewarm 
defenders in all of those who would profit by the new." 
While many of Machiavelli' s methods for solving prob- 
lems were despicable, we must grant him his analysis. 
Since an educational institution is conservative and bound 
by decades of and not centuries of tradition, and is com- 
mitted at least in part to the guild system of master and 
apprentice, a general climate favorable to change must be 
established if a catalytic agent is to have a chance to 
work. Such a climate depends upon developing an institu- 
tional commitment to improvement. As President John 
Hannah of Michigan State has put it, "We see old models, 
old attitudes, old methods, old v alues being challenged and 
changed in society all around us. Can we expect th..* uni- 
versity, itself a social instrument, to escape unchal'enged 
and unchanged?"^ -We cannot. 

Administrative Commitment to Innovation — Far too much 
educational administration involves reacting from crisis 
to crisis in an attempt to keep the educational machine 
operating without major overhaul. There must be ad- 
ministrative commitment to finding better way*3 to solve 
the growing problem.^. This may require an uprooting of 
old patterns of thought and a new willingness to consider 
new proposals and new methods, A commitment to self 
improvement on the part of a university administration 
must be communicated to and be supported by governing 
boards on the one hand and deans and department chair- 
men on the other. 

Talk is not enough. The university must commit some of 
its own financial resources to innovation. Experimentation 
and development cost money. More important, they take 
faculty time. Faculty members must be released from 
their day to day problems so that they can work in depth 
on stimulating new projects. The necessary technical 
support must also be provided and, above all, faculty 
members involved in significant innovation must be given 
visibility and professional recognition for their contri- 
butions to the well-being of the institute. 

Faculty Commitment to Innovation — Faculty commitment 
to innovation must be developed. Academic planning and 
educational development in its truest sense is the province 
and major responsibility of the faculty. Progress in cur- 
ricular and instructional change is almost totally de- 
pendent on university faculty devotion to and concern 
for academic improvement. In the faculty of almost 
every department there are prestiged senior members 
as well as young people who believe improvement of 



^john A. Hannah, ’’Developing a Tradition of Innovation,” An address pre- 
sented at the National Conference on Curricular and Instructional Innovation, 
Mimeographed, November, 1966, p. 3. 
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present procedures and methods is mandatory* 5 Any 
department chairman worth his salt can readily identify 
this dynamic nucleus. When such nuclei are charged 
to examine critical problems and propose new solutions, 
it can be predicted that the commitment to improvement 
will spread throughout the departments and the university. 

Student Commitment to Innovation — Study after study in- 
dicates that thinking students have serious questions about 
the way our universities are run. Students should be 
challenged to come forward with positive proposals. 
Channels must be provided, particularly at the depart- 
mental and college level, to evoke provocative thought 
concerning the relevance and effectiveness of department 
and college programs, and how they can be improved 
within the framework of realistic alternatives. 

A climate favorable to innovation can be developed. It 
may take some patience and some time, but the adminis- 
tration, the faculty, and the students do have the same 
concern; namely, the improvement of the instittttion and 
the quality of the education it otfers. Catalytic agents, be 
they carefully evolved statements of policy, blue-ribbon 
faculty committees, or formally structured agencies, can 
address the real problems and help to develop a favorable 
climate. An example of a formally structured agency — 
the Educational Development Program of Michigan State 
University — is described in some detail below. 

THE EDUCATIONAL DEVELOPMENT PROGRAM 

Scope — 

In April of 1964, the Educational Development Program 
(EDP in local parlance) was formally established with 
support from the Ford Foundation and matching funds 
from the university. The program has been primarily 
restricted to undergraduate curricular, instructional and 
resource development. 6 This decision was and is in line 
with the national concern for the survival of undergraduate 
programs. What are the characteristics of undergraduate 
education at Michigan State? A study revealed the following: 

In the fall term, 1963, 70 per cent of the on-canipus 
undergraduate student credit hours were produced in 
the four departments of the University College, the 
basic teacher training program in the College of Educa- 
tion, and 11 departments. The 11 departments listed in 
order of student credit hour production were mathe- 
matics, psychology, foreign languages, history, English, 
chemistry, economics, sociology, accounting, political 
science and philosophy. Assuming 70 teaching depart- 
ments, 70 per cent of all undergraduate student credit 
hours were produced in 15 per cent of the departments. 
In the fall term, 1963, 40 per cent of the undergraduate 
student credit hours were produced in 50 courses. As- 
suming 900 undergraduate courses, 40 per cent of the 



recent study of the innovators at Michigan State indicates that a large 
proportion of innovative projects comes from outstanding senior members of 
th<* faculty who do not feel threatened by change. Frequently the young 
people feel they must concentrate on * splaying the game** at least until they 
have been tenured and have reached the professorid rank. 

^In the fall of 1964, the total on-campus enrollment of Michigan State was 
31,459(in 1966 it is 38,107). Of these enrollments, 25,963 were undergraduates 
(1966 - 30,753 undergraduates) and 5,496 were graduates (1966 - 7,354 
graduates). This preponderance of undergraduates continuing into 1966 and 
for the foreseeable future reinforced the decision. 



Student credit hours were produced in roughly five per 
cent of the courses. 

In other words, a major segment of undergraduate edu- 
cation was localized. Throughout its brief history, EDP 
has considered and where possible provided support for 
departments and courses not falling in the target area, 
but the heartland of the problem was clearly defined. 

Objectives 

A statement of Educational Development Program objec- 
tives was made shortly after its inception. Today — three 
years later — the objectives remain intact: The Educa« 
tional Development Program will be devoted to the de- 
velopment and implementation of a set of educational 
principles and procedures at Michigan State University 
which will be developed and approved by the general 
facility and which will preserve and improve under- 
graduate education. 

The purposes of the Educational Development Program 
are (l) to identify major problems in the areas :?f cur- 
riculum, the learning-teaching process and the utilization 
of faculty, financial and physical resources; (2) to stimu- 
late and conduct research which will suggest solutions to 
identifiable problems; (3) to undertake projects and studies 
v'hich give promise of improving both the quality and the 
efficiency of the undergraduate program; (4) to support 
and provide service to groups interested in experimenta- 
tion with new procedures and methods in learning and 
teaching; (5) to facilitate implementation of faculty and 
administration approved solutions to problems; and (6) to 
identify and communicate progress in research, experi- 
mentation and implementation. 

Organization 

Since the EDP function is to coordinate, facilitate, com- 
municate and stimulate educational development, there 
was little reason to create an extensive organization. 
Therefore, EDP has not duplicated any organization, 
structure or capability already present in the university. 
It has conserved its modest resources for curricular, 
instructional and resource development projects. The EDP 
office consists of a director, an assistant director, a one- 
half time computer programmer and two secretaries. 
Beyond this small core staff, a number of experts from 
the regular university faculty are supported on an occa- 
sional, part-time, released-time basis to provide neces- 
sary guidance and help xn the implementation of faculty 
designed projects. While not a direct part of the organiza- 
tion, EDP depends on two allied agencies for help in the 
development and servicing of its projects. The Office of 
Institutional Research, which conducts continuing studies 
on the internal operations of Michigan State, provides 
extensive counsel and support in data collection. A new 
agency, the Institutional Development Service, serves as 
an example of a university reorganization proposed by 
EDP. It has three divisions: a Learning Service, an In- 
structional Media Center, and an Evaluation Service. 

The Learning Service collects and applies knowledge 
about the learning process and instructional procedures. 
It serves EDP projects involving faculty members inter- 
ested in defining course objectives, specifying required 
behaviors, exploring new teaching techniques or relating 
test results to teaching practices. The Instructional Media 
Center is responsible for the coordination and develop- 
ment of instructional applications of audiovisual media 
including closed circuit television and the improvement 
through research and development of programs and 
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materials designed for instructional purposes. The Evalua- 
tion Service cooperates with teaching departments in the 
evaluation of student performance and the improvement 
of common term-end examinations. 

The Project Base 

The EDP functions on a project base in much the same 
manner as other funding agencies. Proposals are sub- 
mitted by faculty members, faculty groups, faculty com- 
mittees, departments, colleges, and the administration. 
All projects must have the approval of the appropriate 
department chairman and college dean. Project proposals 
are kept simple. If questions arise, suitable faculty ex- 
perts discuss the proposal with the submitting group. 
Other faculty experts screen the proposal and make 
recommendations concerning its support. Typically a 
well-thought-out proposal can be processed from initial 
discussion to granting in a period of less than tv/o weeks. 

Four general criteria have been established against 
which all projects are measured. These are: (l) The 
number of students affected. In general, EDP is concerned 
with those courses and departments which have large 
student enrollments. (2) Evidence of an experimental 
approach to curriculum or instruction. Proposals which 
merely amplify traditional procedures are referred to the 
departments and colleges for support. (3) Potential gen- 
eralization to other academic areas. Projects which are 
so specific or narrow as to have little applicability to 
other parts of the university are generally refused. 
(4) The possibility of evaluation. Procedures for evalua- 
tion are built into all projects. 

The EDP supports projects through the experimental 
phase. Upon the successful completion of a project, EDP 
recommends that the university funds nece«^^ry to carry 
on the innovation be placed in the appropriate depart- 
ment or college budget. 

Levels and Areas of Operation 

The EDP works at the levels of university policy college 
and departmental operation, and individual student learn- 
ing. Simultaneously, it works with curriculum, instruction 
and resources at each level. Seventy-five projects have 
been conducted within this framework and are distributed 
as shown below. 

DISTRIBUTION OF SEVENTY- FIVE EDP PROJECTS 
BY LEVEL AND AREA (1966-67) 



LEVELS 




DEVELOPMENT AREA 






Curriculum 


Instruction 


Resources 


University 


8 


8 


6 


Department & College 


6 


15 


6 


Student 


9 


8 


9 



The projects range from participation in the development 
of the new cluster colleges to total revision of the univer- 
sity curricular guidelines, from evaluation of college 
organization and curriculum s to depth studies of individ- 
ual departments,^ from development of new multimedia, 
structured, independent learning- and- teaching environ- 
ments to the redirection of direct-instructional, closed- 
circuit television, from reorganization of the procedures 



^The rationale and method for departmental study is reported by Paul L. 
Dressel and John 2. Dietrich, “Departmental Review and Self-Study,** /ouma/ 
of Hiihet Education, Vol. XXXVIII, No. 1, January, 1967, p. 25-37. 



for credit examination to improvement of examinations 
in more than 30 large courses. 8 

Initial Assessment of the EDP 

Recognizing the number of areas and levels in which 
EDP has worked, it is difficult to assess with any degree 
of certainty the amount of change directly attributable 
to the program. Without question some of its accomplish- 
ments must be attributed to the ^^institutional environ- 
ment*' which it has helped to develop and within which it 
works. 

Judging success — At least four criteria may be used for 
judging the program. The first criterion is the frequency 
and degree .of participation in the major educational move- 
ments within the university. It can be demonstrated that 
EDP has provided service and support in connection 
with most of the recent changes occurring within the 
institution. A second criterion is the degree to which 
innovative ideas have moved from department to depart- 
ment. Numerous instances can be cited to show that 
successful developments in one department have been 
transferred where appropriate to other departments. 
A third criterion is the positive results accruing from 
intensive evaluation of individual projects supported by 
the program. These evaluations of both learning and 
student attitude clearly indicate success in a number of 
areas,? A fourth criterion might be called the **multplier 
effect." In the three years of formal operation, the 
number of project requests have quintupled and have 
given evidence of increasing at an even greater rate. 
Measured against these kinds of criteria, the EDP can be 
considered a succesSi 

Judging failure — While the success of the EDP appears to 
be significant- it is also important to recognise that the 
program has had its failures. These are failures by 
omission and failures of commission. 

There are significant failures by omission. Some de- 
partments in the university have not sought the help or 
support of the .program. Subjective judgment of this 
failure leads to the conclusion that the willingness to 
consider innovation is related to the felt need to solve 
problems. Many faculty members apparently feel no 
need to consider new or improved methods if traditional 
patterns seem adequate. If the number of faculty and 
staff is adequate, if the technical resources are sufficient, 
if the class section size is reasonably small, and if the 
vocational and professional accrediting obligations are 
met, there is little motivation to scrutinize present 
practices with an eye to improvement. 

There have also been failures of commission. Some- 
where between six to eight per cent of the 75 EDP 
projects have been failures. Several of these failures 
occurred in the initial stages of the program and probably 
resulted from a lack of efficient and organized screening 
and evaluation of the proposed projects. Others repre- 
sented poor judgment on the part of the project developers 
and the EDP directorate, and still others failed because 



report which gives an abstract of each project may be obtained from the 
Educational Development Program, Administration Building, Michigan State 
University, East Lansing, Michigan 48823. 

^Reports of specific evaluation studies in such areas as Closed Circuit 
Television, Structured Learning and Teaching i^nvironments (SLATE labora- 
tories), or the Co-Curricular Program, may be obtained from the Educational 
Development Program, Administration Building, Michigan State University, 
East Lansing, Michigan 48823. 



of inadequate faculty commitment. However, any innova- 
tive, experiment-oriented agency must have not only ''the 
right to fail," but also the willingness to withdraw failing 
experiments quickly. 

Dangers of a Catalytic Agent 

It is easy for any agency responsible for innovation to 
become embroiled in "innovation for innovation’s sake" 
or to introduce change in inappropriate ways. As industry 
has illustrated time and again, new ideas, products, and 
methods can be undertested and oversold. If faculty ad- 
visers and research experts are used on all projects, it 
is easier to avoid this pitfall. Careful screening, constant 
liaison, and intensive evaluation help to bring objectivity 
rather than subjectivity, logic rather than persuasion 
into the area of innovation, 

POSSIBLE APPLICATIONS TO 
OTHER INSTITUTIONS 

It would be gratuitous to suggest that Michigan State 
University’s EDP could or should be transplanted directly 
in all of its aspects to any other institution. Each college 
and university has its cwn special characteristics and 
style. On the other hand, EDP has certain unique charac- 
teristics which we believe are basic to its success. These 
characteristics can be developed in any institution regard- 
less of size. Our charge to any administrator considering 
the development of a structured catalytic agent would be 
as follows; 

1. Establish a small directorate . An educational develop- 
ment program exists to stimulate, facilitate and com- 
municate, There is no need for it to become an empire, 
A small directorate of one or two people will be sufficient 
to coordinate the largest program. 

2, Provide an overview of academic problems. The best 
overview is found in central academic administration. 
Sooner or later almost all problems land on the desk of 
the Provost or Dean of Faculties, The director of the 
program should have regular contact with the chief 
academic officer. 



3. Give access to key faculty committees. Many of the 
problems the program will be asked to help solve will 
arise in faculty policy and curriculum committees. Not 
only must the director understand the faculty point of 
view, but perhaps more importantly, the faculty must have 
confidence that the director understands their point of 
view. Furthermore, these groups will frequently be part 
of the channel through which solutions must flow. 

4, Coordinate existing expertise. Often the testing, media 
and learning experts and even the institutional research 
experts on the campus are working unilaterally to develop 
their own facilities. In some instances they may be con- 
sciously or unconsciously competing with or at least 
duplicating each other. They may even be aware of the 
institutional problems which require their special skill. 
Coordination of these experts can provide solutions to 
real university problems. If additional expertise is neces- 



sary, it should be placed in these groups rather than 
expanding the directorate, 

5. Provide discretionary funds. Many times a small 
amount of money can help solve very large and real 
problems if the money can be committed quickly. Other 
times, large and costly projects can be given "seed" 
money until external support can be found, A principal 
obstacle to innovation is the shortage of faculty time, Ey 
the use of released time, faculty members can be freed 
to work intensively on new ideas. Further, discretionary 
funds can be used to encourage action-oriented research 
on immediate problems. Thus discretionary funds make 
possible the mounting of immediate faculty action, 

6. Build a grant procedure within the university. A project 
base gives the chance to select the activities which most 
need support. A simple proposal, review, approval, 
monitoring and reporting function should be established. 
Faculty members should spend only a minimum time on 
this procedure and devote a maximum effort to the project 
itself. 



7. Encourage faculty to submit proposals. Most problems 
can be solved only by the faculty most directly concerned. 
The small directorate neither can nor should take an 
active part in projects. 

8, Provide continuing liaison with projects. Projects 
should not be funded and forgotten. Continuous liaison 
should be supplied from inception to completion. In some 
instances when departments or colleges have several 
on-going projects, faculty members may be appointed to 
serve this liaison function. 



9. Build in evaluation. Experiments tend to become per- 
petuated in the system — sometimes regardless of worth. 
Failing experiments must be eliminated. Evaluation should 
be a part of each project. Often the faculty involved is 
best able to do the evaluation while at other times evalua- 
tion by an external agency may be desirable, 

10. Establish regular university support for successful 
projects. All projects should be reviewed. Those judged 
to be successful should be continued in the regular uni- 
versity operation and supported from regular university 
funds. 



Regardless of whether the methods used by Michigan 
State are applicable to any other institution, the problems 
are here, the dilemmas exist, the questions remain un- 
answered. John Gardner has expressed it well in the 
concluding note to his "Agenda for Colleges and Uni- 
versities:" 



I do not believe that the colleges and universi- 
ties will go under because they are carrying 
heavy burdens. If they deteriorate, it will be 
because they lacked the morale, the internal 
coherence, and the adaptiveness to meet the 
requirements of the future; it will be because 
in the moment of their greatest success they 
could not pull themselves together to face new 
challenges. 
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James L. Miller, Jr. 

I 1 

We are living in a time of revolution. Yesterday's New 
York Times carried feature stories about the over- 
throw of the Russian Czar just 50 years ago. It seems in- 
credible that it was so short a time ago — less than a 
lifetime. Revolution came violently to Russia, and in the 
years since, revolution has come violently to a long list 
of nations. Some have suffered repeated revolutions, 
complete with destruction, pestilence, and often with an 
aftermath of broken homes and tyrannical suppression 
of those who dared to remind their countrymen of the 
visions of progress which first kindled the revolutions. 
Throughout the world man is struggling toward his dream 
of a better world, and his struggles all too often lead 
only to disillusionment. 

It is paradoxical that in this age of revolution it is in the 
United States that one finds the most sweeping revolutions 
of all. No place on the earth has been transformed more 
completely, no people's way of life has been changf^d 
more basically, than in the United States. The United 
States has witnessed not one revolution, but a whole 
series of them and many of them have gone on simul- 
taneously. They go unrecognized for the revolutions they 
are, because they are accompanied by little or no violence. 
I believe that this is attributable to the fact that the basic 
cause of revolutions in the modern world is the rising 
expectations of the general population, and the United 
States has found ways in which to meet those rising 
expectations. As a result, we in the United States have 
enjoyed revolution without the Evolution is less 

dramatic on the surface, but in our case, at least, we 
have demonstrated time and again that the changes which 
it can bring are fundamental. 

The Multiversity and the Community College 

The most important single instrument of evolutionary 
revolution in the modern world is education. In contem- 
porary America two of the most dramatic manifestations 
of revolutionary education are the multiversity and the 
community junior college. This conference brings to- 
gether representatives of both to discuss the extent to 
which they share a common cause and the ways in which 
they can better articulate their efforts. That such a 
meeting should take place, that it should take place 
annually, and that its occurrence should create no sur- 
prise is the most eloquent testimony possible to the 
American acceptance of evolutionary change and of simul- 
taneous multiple revolutions. Indeed, it provides evidence 
of the further fact that many of America's revolutions 
are pragmatically interlocked and mutually supportive. 
I think we Americans fail completely to grasp the broad 
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significance of such an approach to meeting the crisis 
of rising expectations. 

The revolution to which the multiversity is the key is the 
emergence of knowledge as the basic element for the 
continued development and good health of the post- 
industrial society into which we are fast moving. The 
revolution to which the community junior college is the 
key is the mass utilization of knowledge — for occupational 
purposes and for personal ones. The multiversity, through 
research, opens the doors to the future; the community 
junior college, through universal higher education, leads 
the American population into that future. The process, 
and the results are the process, are the envy of the wox^ld. 

Center for the Study of Higher Education 

I trust that I have painted with sufficiently broad strokes 
to vsuggest that I think there is an import to this confer- 
ence which far exceeds its specific purpose and content. 
This is one of the things we try to accomplish at the 
Center for the Study of Higher Education — the placement 
of specific aspects of the here and now in their larger 
historical and philosophical context. For students in the 
Center the basis for this is more than a few minutes 
of painting with the broad brush; we try to provide a 
firm grounding in the form of solid course work in the 
history and philosophy of higher education. 

We also try to do something else at the Center. We try 
to fill in the larger picture with the details of the what 
and Why that will enable our students to deal construc- 
tively and realistically with the specific pieces of the 
higher education scene with which they will be working. 
Without losing sight of the total picture, we try to pre- 
pare people for administrative positions in institutions 
of many types. Among the many aspects of higher educa- 
tion with which we try to deal, some of our students 
specialize in community junior college work, and some 
specialize in the functional activity of institutional re- 
search and planning. 

Institutional Research 

Institutional research is a relatively new addition to the 
family of administrative positions. The term institutional 
research is misleading because of the easy confusion 
between it and on-going programs of academic reasarch 
within institutions, but we seem stuck with it. A better 
term for it would be management research or analytical 
studies, either of which much better denotes the thrust 
of the enterprise — the improvement of the operation of 
individual institutions or cooperating groups of institu- 
tions through careful study and analysis of their organi- 
zation, and their operational processes and outcomes. 
These include not only those associated with such clearly 
administrative activities as budgeting and space utiliza- 
tion, but also those that get closer to the heart of the 
operation such as studies of admissions, teaching, and 
inter-institutional articulation. 

I fear that most of us who have some familiarity with 
institutional research still fail to grasp the full breadth 
of the field because of the fact that most institutional 
research offices tend to concentrate upon one or another 
specific group of institutional research activities. This 
is due in part to the particular interests and capabilities 
of the institutional research staff in various institutions, 
and in part to the particular needs of individual institu- 
tions at a particular time. The specific needs and in- 
terests which cause the establishment of an institutional 
research office in the first place often acts to determine 
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the type of staff employed and the focus of attention for 
far longer than needs to be the case. 

lndeed> there often develops v/ithin the institution the 
assumption that the special interests of that institution's 
institutional research office is the proper sphere of in- 
terest of institutional research in general. As a result 
I have seen situations in which institutional research 
offices which originally concerned themselves with space 
utilization studies and financial analysis were told, or 
told themselves, that it would be inappropriate and im- 
proper for an institutional research office to concern 
itself with such things as studies of student admissions, 
student chracteristics, student performance, teaching 
techniques, and faculty characteristics; and by the same 
token I have seen institutional research offices which 
started out by making studies of student related activities 
that subsequently refrained from entering into '*inap- 
propriate’* activities such as studies of space and costs. 

It is interesting to see some of these people assemble 
the annual meetings of their national organization, the 
Association for Institutional Research. The initial shock 
sonie of them experience at the variety of their brethrens' 
interests is instructive. F^w people leave such a meeting 
without developing a broader concept of what the term 
institutional research can encompass. 

A Recent Phenomenon 

I emphasize this point because I know that many of your 
institutions have relatively recently entered into institu- 
tional research. The self-conscious designation of in- 
stitutional self-study as a continuing process requiring 
the full-time energies of one or more specialized staff 
people is a recent phenomenon. With the exception of 
only a few institutions, it is a post World War II phe- 
nomenon, and for the majority of institutions it is a de- 
velopment of the last five or 10 years. The term institu- 
tional research has been in general usage for less than 
10 years, the convening of annual national conferences 
on institutional research is an occurence of less than 
10 years duration, and the national organization of insti- 
tutional research officers is only two years old. The 
establishment of institutional research offices in junior 
colleges was rare a few years ago, but they are becoming 
more and more common. 

Because institutional research personnel come from 
such a variety of backgrounds, and because the offices 
themselves have been established for such a variety of 
reasons, I think there is a tendency for most of us to 
miss the central significance of institutional research — 
the fact that its distinguishing characteristic is the 
systematic application of analytical techniques to issues 
and problems associated with the organization and opera- 
tion of a college or university. Colleges and universities 
have accepted the basic concept that it makes sense to 
apply to their own operations the same type of rational 
processes which they have helped bring to bear upon 
other social organizations such as business and govern- 
ment. The acceptance of this concept means that college 
and university administration has moved out of the area 
of intuitive, and into the area of the rational. It does not 
mean that there is no artistry left in the administrative 
process; it does mean that the administrative artist is 
provided with the best analytical tools, and his artistry 
is exemplified not by how well he can guess at the facts, 
but how well he can utilize the facts, interpreting them, 
i*iter-relating them, orchestrating them, to produce the 
most harmonious possible ensemble. 



Facilitating Student Flow 

This meeting is concerned with certain inter institutional 
aspects of our utilization of our knowledge about students. 
Of central concern is facilitating the flow of students 
from community junior colleges to senior institutions. 
The first prerequisite to this process is the acquisition 
of basic information about stUvdents and about programs 
by each of the institutions individually. This information 
is not gathered by them only for the purpose of inter- 
institutional cooperation, since is is needed even more 
for a variety of intr a- institutional purposes, but it never- 
theless is essential to inter- institutional cooperation. 
Without it, there is no common vocabulary, no certain 
set of facts with which to begin. 

Not too many years ago the elements of information 
involved were fairly few and fairly simple — a few student 
test scores and some basic information about course 
offerings within the institution and student achievement in 
those courses. Today the information potentially useful 
is far more complex. The new ETS college level tests 
will add a major new dimension to our measurement of 
student achievement. The proper utilization of these tests 
will require an adequate consideration of some institu- 
tional issues which go beyond the test results themselves. 
Are they to be used as a supplement to other achievement 
measures, or as a substitute for them? If they are utilized 
in the determination of eligibility for transfer between 
institutions, what does this imply for a university' s obliga- 
tion to utilize them in determining the retention or non- 
retention of native students to be enrolled there in the 
first place? 

Student Admission 

Initial admission to most colleges and universities has 
been based primarily upon records of actual achievement, 
of potential for achievement or some combination of the 
two. So has transfer between institutions. I think there is 
general agreement in higher education that for most 
institutions these are proper bases for admissions deci- 
sions, Exceptions are those institutions that receive more 
applications from high achievers than they handle, and 
''opportunity" type admissions, which constitute a form 
of compensatory opportunity which is afforded in an effort 
to help overcome the effects of unusual deprivation. 

Stated somewhat more globally, admissions decisions, 
whether concerning initial admission or transfer admis- 
sion, should relate the probability that the student will 
benefit from the program of the institution to which he is 
transferring; for most institutions the most impartial 
measure of this can be stated in terms of student achieve- 
ment-past or potential. How tightly this is defined varies 
markedly among institutions; this is one of the desirable 
aspects of the diversity which characterizes American 
higher education. 

Admissions, however, involves more than the question of 
whether a student is eligible for admission. It also in- 
volves, or should involve, the question of whether a stu- 
dent should be advised to enter a particular institution. 
Stated another way, which institutions should a particular 
student be counselled toward, and which institutions 
should he be counselled away from. The virtual deluge 
of information which has begun to appear about student 
characteristics and about institutional characteristics 
that affect student performance and student satisfaction 
has opened a whole new set of questions which counselors 
and institutional administrators in general will be wres- 
tling with for a long time. 
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The work of such men as Ted Newcomb and Jerry Gurin, 
of The University of Michigan^ has contributed greatly to 
our understanding of students, student behavior, and the 
variation in campus environments, and climate, as has 
the work of researchers elsewhere like Robert Pace, 
George Stem, Martin Trow, Burton Clark, Nevitt Sanford, 
Dorothy Knoell, Leland Medsker, and others. 

One piece of information of particular significance to this 
conference is the accumulating evidence that students 
who do well at complex multiversity campuses are stu- 
dents who have personality characteristics which prepare 
them to live successfully in the relatively impersonal 
setting of the city-big campus. For example, Ann Arbor 
is not a big city, but in many respects The University of 
Michigan and Michigan State University in East Lansing 
have the social and psychological impact of the city. The 
student who transfers from a small campus to these 
universities experiences many of the same shocks ex- 
perienced by the small town resident who moves to the 
larger city. 

Our cities are populated by people who have moved from 
the small towns, so there is no question about the ability 
which large numbers of people have to successfully make 
the transition. We also know, however, that many people 
do not make the transition successfully, and we are 
learning more and more about ^he social and personal 
characteristics associated with success and non-success. 

Questions to Consider 

What is the responsibility of the counselor to familiarize 
himself with these characteristics, to look for them in 
the students he counsels, and to be guided accordingly? 
This point can be pressed still further. America rapidly 
is becoming an almost totally urban society and, in keep- 
ing with the times, most of our senior colleges and uni- 
versities are becoming larger and much more complex. 
What is the obligation of the community junior college to 
examine its own program with the question in mind: 
What are we doing to prepare our students for life in a 
world which inevitably will be big almost to the point of 
being overwhelming? And what is the responsibility of the 
university to assist students in the transition to that kind 
of world, both in its on-campus and its off-campus or 
post-university aspects? The University of Mic! 'gan's 
^'Residential College*' is one such attempt, but it ob^ ously 
is only one piece in a very large puzzle. 

I think that these questions concerning the student ad- 
justment to bigness — bigness in our colleges and uni- 
versities and bigness in the world outside our colleges 
and universities — and adjustments to the world tomorrow 
will be and should be matters of major concern to both 
community colleges and universities for many years to 
come. In many respects these questions, and the institu- 
tional answers to them, are intimately associated with the 
necessary inter-relationships which must exist between 
community colleges and universities. They are, therefore, 
a proper concern for this conference in this and future 
years. 

As I indicated earlier, we live in revolutionary times. 
America, as a nation, has found the answer to harnessing 
the power of its multiple revolutions to satisfy the rising 
expectations of its people. One of the key factors in :his 
process has been the cooperative interaction between such 
revolutionary forces and the multiversity and the com- 
munity junior college, both of which are represented here 
today. 
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I would like to do two things: first, to discuss some 
relatively new instruments, now in various stages of 
development or refinement, and to describe activities for 
community or junior colleges in which our organization 
is engaged either independently or as a partner; and, 
second to suggest some elements of institutional research 
which, I consider important to you. 

Before continuing I should like to offer two disclaimers. 
First, we don't have a "program" for institutional re- 
search or evaluation, if by a program is meant an in- 
felxible, rigid package of tests and services. Such is 
antithetical to the research process and to what is known 
about the diversity of needs in your institutions. Seriously, 
I claim no particular expertise concerning junior or 
community college institutional research. In fact, most of 
us who are or have been involved with institutional re- 
search or who otherwise bring to this problem area 
backgrounds in measuremCi^'t and psychology are quite 
unfamiliar with the "proble’us of the junior or community 
college'' — and we are taking whatever steps we can to 
become more conversant with such problems. Having made 
these disclaimers, I shall proceed to the tasks at hand. 

Presently, at the Educational Testing Service, we have 
two fairly-well developed instruments in use, both may be 
interesting to you. ETS did not develop either of these but 
served as the agency through which data were assembled, 
revisions were made, and studies using these instruments 
were conducted. The first instrument, with which many 
of you are familiar, is The College and Uriversity En- 
viromental Scales, known as CUES, and developed by Dr. 
Robert Pace. The second is The College Student Ques- 
tionnaires which resulted from the work of Dr. Martin 
Trow. Both are group instruments — that is, the main 
purpose of each is to collect data on groups as groups. 
Their objective is not to obtain information about indi- 
viduals, though the individual is the medium through 
which the group information is obtained. 

The present CUES began several years ago as a result of 
the joint efforts of Drs. Robert Pace and George Stern, 
then both of Syracuse University. Their efforts yielded a 
300-item instrument, known as the College Characteristics 
Index (CCI), based on a personality configuration of 30 
variables developed by Murry and predicated upon his 
concept of "environmental press." Stern's interests in 
the instrument was focused upon che individual and his 
personality, while Pace's interests were directed toward 
educational practice and the description of educational 
environments. Some five years ago. Pace left Syracuse 
for UCLA. There he used the data gathered with the 300 
items of the CCI, chose the 150 items which best discrimi- 
nated or highlighted the difference between institutions. 
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and assenmbled the 150 items into sets of 30, yielding 
five scales and the instrument known as CUES. 

The five dimensions or scales of CUES are: (1) Practi- 
cality: the degree to which personal status and practical 
benefit are emphasized in the college environment. Status 
is gained by knowing the right people, being in the right 
group, and doing what is expected; (2) Community: the 
degree to which the campus is friendly, cohesive, and 
group oriented; (3) Awareness: the degree to which there 
is concern with self-understanding, reflectiveness', and the 
search for personal meaning; (4) Propriety: the degree to 
which politeness, protocol, and consideration are em- 
phasized, and (5) Scholarship: the degree to which serious 
interest in scholarship and competition for academic 
achievement are present. The purpose of this instrument 
is to describe the ‘^institutional press'' or the environ- 
ment of the institution by means of these five scales. 

In general, use of the instrument during the past few 
years has served two broad purposes: (l) to provide 
additional information about undergraduate institutions to 
improve the guidance of secondary school students, and 
(2) to provide data for institutional planning by adminis- 
tration and staff of undergraduate colleges and univer- 
sities, including in some cases attempts to manipulate 
certain physical, social, and program features of the 
institution amenable to manipulation and change. 

The description of the educational climate provided in the 
catalog issue of the Antioch College Bulletin and the 
environment information on some colleges found in the 
Manual of Freshmen Class Profiles of Indiana Colleges 
are examples of attempts to use CUES as additional 
guidance information. More rare are examples of attempts 
to study and then to change the environtneniral press of an 
undergraduate institition. However, a personal experience 
might be apropos to this discussion. 

Five years ago, while serving as director of institutional 
research at St. Louis' Webster College, we became dis- 
tressed, after validating student responses to CUES, by 
our position on the CUES scales. We considered the de- 
scription valid but inconsistent with certain institutional 
goals which were then undergoing rapid change. One s.xh 
scale position with which we were unhappy was our high 
position on the propriety scale — not that we were against 
high propriety — but we were against how we got the high 
position. We found certain critical and regulatory “press" 
oppressing to our students, a group which was then more 
liberal than those who had attended the institution in the 
past. We decided that this “press" was inconsistent with 
certain goals of the college, and we set out to change 
things. Tight control appeared to us inconsistent with our 
desire to have Webster College women see themselves as 
adults capable of addressing important social, political, 
and theological issues of the day. For literally a pound of 
rules, regulations, instructions, etc., sent through the 
mail to the high school seniors we had accepted for ad- 
mission, we substituted a letter of welcome and a state- 
ment to the effect that they would be treated as young 
adult women until and unless we found behavior expressed 
to the contrary. For nuns and other types of hall monitors, 
we substituted the check of individual responsibility. 
Instead of lights-out regulations, an elaborate system of 
control of permissions to leave campus, and certain 
smoking restrictions, we again relied on the individual's 
sense of responsibility. There was a noticeable change. 

The second instrument I wish to mention is The College 
Student Questionnaires (CSQ). These questionnaires have 
been designed to obtain descriptions of students primarily 



in terms of biographical and attitudinal characteristics. 
The questionnaires also include questions concerning stu- 
dent's actitivies, perceptions, and staisfactions as stu- 
dents at a particular college. The instrument consists of 
two separate questionnaires — Part I and Part II. Part I 
is administered to entering students; Part II is adminis- 
tered to undergraduates at the end of each academic year. 
Each questionnaire contains 200 multiple-choice questions 
and requires approximately 60 minutes for completion. 
Together the two parts of the CSQ contain 13 scales 
consisting of 10 items each. Five of the scales are com- 
mon to the last section of both questionnaires. The scales 
are named Motivation for Grades, Family Social Status, 
Family Independence, Peer Independence, Liberalism, 
Social Conscience, Cultural Sophistication, Satisfaction 
with Faculty, Satisfaction with Administration, Satisfaction 
with Major, Satisfaction with Students, Study Habits, and 
Extracurricular Involvement. Again it should be noted 
that the questionnaires are appropriate only for suggesting 
possible differences among groups, not individuals. 

Both of these instruments, along with other achievement 
measures, are available through a flexible plan of instru- 
ments, services, and consulting, (if desired), called the 
Institutional Research Program for Higher Education 
(IRPHE). 1 

With regard to the measurement of personality, an area of 
concern and interest to you, I am less optimistic that 
available instruments from any source will have practical 
application for you in the near future. Research staff at 
ETS have been involved with personality research for the 
past 10 years or more. Thus far we have available for 
research purposes only the Myers-Briggs Type Indicator 
from our Office of Special Tests. ^ The measurement of 
personality still has a number of sticky construe tural, 
procedural, and technical problems wh?ch will probably 
remain with us for a long time to come. 

Now I would like to turn to a brief description of some 
ETS activities in behalf of junior colleges. As a part of 
this description I shall mention some newer instruments 
which are in various stages of development. During the 
past year, the American Association of Junior Colleges 
and ETS have conducted several meetings to discuss co- 
operative efforts and joint projects in an attempt to meet 
some of the problems of measurement of junior college 
populations. Recently these two groups have issued a 
charge or a mandate which created an advisory committee 
to ETS as joint efforts are planned. 

The first broad effort will direct attention to specific 
instruments and/or programs in the vocational- occupa- 
tional area. The advisory committee will (l) define the 
broad concerns of the p*^oject and consider appropriate 
dimensions for implementing objectives; (2) provude a 
medium for developing plans for continuing review and 
discussion for the measurement program; and, (3) assist 
in establishing adequate communications with the jur..: & 
college community. 

Thus far I can report four tasks which have been initiated: 
(l) the undertaking of a descriptive survey of the junior 
college population; (2) the developing of a program and 
the materials for improving the training of junior college 
student personnel workers; (3) the developing of measures 

^Further information about IRPHE or IRPHE instxumrnts can be obtained by 
contacting Dr. Eldon C. Park, IRPHE Program Director. Educational Testing 
Service, Princeton, New Jersey 08340. 

Those interested in details of this instrument may contact Mr. Francis X. 
Nulty, Director of the Offic of Special Tests, Educational Testing Ser^'ice, 
Princeton, New Jersey 08540. 
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to describe the outcomes of occupational programs; and, 
(4) the developing of techniques to measure the special 
characteristics, interests, and aptitudes relevant to an 
individuaFs entry into occupational programs,^ 

In addition to AAJC-ETS activity, the College Board has 
begun dialogue with the junior college community and has 
also initiated an interesting project. The interest of the 
College Board in junior and community colleges, of course, 
stems naturally from the Board's interest in its own mem- 
bership as well as the implications of the junior college 
as a means of access to higher education. In that sense the 
Board is concerned not only with the transfer problem, 
but with the transition from school to college, an interest 
and a concern which is a logical outgrowth of its charter. 

Last fall nearly 50 junior college and community colleges 
in all regions of the country we*"e visited and surveyed by 
College Board and ETS personnel to obtain assistance in 
reacting to an experimental program of study that the 
College Board had planned. As a result of this detailed 
survey the College Board will begin an experimental 
program of tests and services in perhaps 50 institutions 
in the next academic year to study how effective certain 
new instruments are and how suitable and valid they may 
be for institutions primarily concerned with the first two 
years of higher education. At present this experimental 
study and the battery of instruments to be studied has 
been given the name of The Comparative Placement and 
Prediction Battery. Basically this study has its instru- 
ments organized into three components. The first com- 
ponent is a core program or a core set of instruments 
measuring academic ability, reading, and mathematics 
fundamentals for placement purposes, with the possible 
addition of a non-verbal ability measure which could be 
associated with occupational areas. Each of these instru- 
ments will, of course, be shorter than the Board's SAT 
or the American College Testing Program's ACT. There- 
fore, each will be less reliable in a statistical sense, but 
the problem will be to provide adequate reliability while 
giving more specific information than either of the widely 
used general purpose instruments. These instruments, 
then, will address themselves directly to providing place- 
ment information. 

The second component of this project will contain interest- 
ing new instruments, like background and experience 
questionnaires, independent activity questionnaires or in- 
ventories, and, although they are related in rationale to 
CSQ, the set will also include certain kinds of special 
instruments, such as tests of cognitive style, in hopes 
that some of thei^ will be found useful for some of your 
problems. The third group or component of the study 
instruments will be those referred to as research instru- 
ments. These include an environmental measure, special 
factor tests of the kind related to what we refer to at ETS 
as the factor kit. a set of instruments developed in col- 
laboration with outstanding psychologists outside of ETS 
over the past 15 years or so. It may very well be that one 
or more of these research instruments will provide real 
payoff in assisting you with institutional research prob- 
lems in the future. A year from now we shall be in a 
better position to say. 

In addition to these activities ETS is presently cooperat- 
ing with individual junior college institutions as well as 
one to two states in working cooperatively to improve 



^Related but distinct from AAJC-ETS efforts is the work of Dr. Dean Seibel, an 
ETS staff member, who surveyed systemically junior college problems and 
needs. A report of his work is available through the Evaluation and Advisory 
Service, Educational Testing Service, Princeton, New Jersey 08540. 
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the measurement of junior college students in these 
institutions."^ The activities just mentioned will give you 
a feeling for the extent of our involvement in matters 
relating to the needs and interestsof junior and community 
colleges. 

Let me touch further on the idea of ''payofP' for you. 
Implicit in the comments which follow, and supporting a 
point made in one of the morning presentations, is the 
assumption that the heart of the entire educational enter- 
prise is the instructional program. Every other institu- 
tional aspect, facility, or source of energy in the last 
analysis exists to support the educational program offered. 
Therefore, much of the focus of institutional research 
should - be directed toward improving the educational 
program. 

A first notion logically follows this assumption. In planning 
improvement you first have to know what you are, or what 
you do, before you can decide what you wish to do, or 
where you wish to go. That is, you must assess the present 
in order to plan the future well. So as increasing numbers 
of institutions in higher education set about the tasks of 
gathering data for policy decisions affecting the future of 
the institution, they find the first effort in institutional 
research is descriptive in character. 

What kind of program, by whom is it offered, how is it 
offered, with what is it offered, and to whom is it offered 
are fairly broad questions which are generally raised. 
Some of the research data reported this morning by Dr. 
Johnson were the result of focusing Michigan State energy 
on some of these questions. 

With even tentative answers to these questions, and the 
data which provide the answers in hand, some judgment or 
evaluation (assigning value to the relative importance of 
information) will result. These evaluations, the values 
assigned, will result in certain satisfactions and dis- 
satisfactions. Both generate ideas for improvement, and 
suggest more specific questions to pursue. 

Although these activities just mentioned are necessary, in 
my opinion -they are also loadedwithtraps.lt is fair to say 
that much of what might be termed institutional research 
has little practical payoff either because it is actuarial or 
demographic in nature or because it is done without 
regard to the broader institutional context. Let me illus- 
trate an example of each type to clarify this statement. 

Every year many studies appear which document credits 
earned, hours by subject, faculty salaries, etc., and every 
year validity studies appear which describe the relation- 
ship between test scores and GPA's. I believe that these 
studies are necessary at times but not sufficient for any 
direct improvement in the instructional program. The 
best we can hope for in a validity study procedure are 
validities in the sixties, which, I would remind you, 
indicate that the test score or scores in combination 
account for well below half of the variance in grades. 
Usually predictors account for a great deal less than that. 
Demographic and "continuing resources" type of institu- 
tional study are not bad in themselves either, but to be 
useful they must be related to some demonstrated effort 
on the program and particularly on a student in the pro- 
gram. This last comment also applies to a final example. 



^Dr. George M. Barton is the ETS staff member who is the Program Director for 
independent ETS activities with junior colleges. Dr. Barton is in the Princeton 
Office. 




Institutional research involving studies content, facilities, 
staff utilization, various media of instruction, and other 
facilitating energy, while often methodically and tech- 
nically sound, remains valueless because the purpose or 
objectives for the activity remain obscure, if in fact they 
were ever known. If the objectives for such an activity are 
unclear or obscure, it is impossible to call up criteria 
which can be used to assign value to the study. In technical 
jargon we call this the '^criterion problem/' My own view, 
then, is that there is one critical problem area which 
transcends in importance all others, and without attention 
will severly limit the effectiveness of any institutional 
research effort. This is the matter of educational ob- 
jectives. 

As long as educationalobjectivesremain vague and couched 
in philosophical terms, or in terms of lists of content to 
be covered, or in terms of what teachers do rather than in 
terms of what students do, we will never realize the 
promise of institutional research. In technical jargon we 
refer to this task as "expressing educational objectives 
in behavioral (operational) terms," In expressing them in 
such a form there is only one referent— -the student. 

I do not have time to develop this idea of behavorial ob- 
jectives further except to underline its importance and in 
all fairness to say that it is a most difficult task indeed, 
A useful beginning with your staffs would be to expose 
them to the best first step I have found. And that is, a 
small but important book by Robert Mager entitled 
Preparing Instructional Objectives. 5 

Once the matter of objectives is addressed and some 
success in expressing objectives in that form has been 
realized, for me three things happen. 

First, operational terms now exist as objectives which 
can serve also as criteria for studies of special kinds. 
Questions concerning which academic calendar we should 
have, whether we should have a large group instruction or 
not, the worth of a new math sequence, or a new course, 
etc,, (the facilitating questions) can be answered with some 
confidence, if in the beginning we know more clearly the 
kind of student behavior or performance desired. 

Second, the placement problem takes on a new look for me, 
when I know from the past record of a student what im- 
portant things he can do in basic mathematics and the 

^Mager, Robert F. Preparing instructional Objectives. Fearon Publishers, Inc., 
Palo Alto, California. 



requirements of next courses in a mathematics sequence— 
as well as when the emphasis is placed on the fit between 
the two. Unfortunately student grade information is of little 
help, nor are total test scores. The problem with both of 
these kinds of information is that they fail to communicate 
what the student can do. 

The placement problem is related to the general criterion 
problem mentioned before in connection with the limita- 
tions I saw in validity studies. GPA's are in a sense more 
reliable as an over-all estimate of pupil quality than one 
course grade. Both the GPA and the grade, however, are 
inaccurate as criteria for the same reason that grades 
and total test scores are inadequate as placement infor- 
mation, They disguise more than they disclose. What 
must be disclosed is w!;at students can now do. 

Certainly we have gotten tests about as reliable as we can 
get them. Although they are still fairly gross measures 
and are likely to remain so, we can look only to the cri- 
terion end of the prediction problem to increase the 
accuracy of our predictions. Expectancy tables (the sta- 
tistical approach) have been with us for a long time. What 
we need is a sharper criterion— a fresh look at the prob- 
lem — a common currency with which to communicate 
pupil performance. 

I would hope that you could encourage staffs to begin the 
development of such a communications system and not to 
fall into the trap of researching everything in the same 
old way as we have done before. 

There are instruments around with junior college names 
on them now and are likely to be more— that fact could 
also be a trap. Tests deserve content analyses with em- 
phasis on what the student does as he answers the items 
to see if the requirements of the test relate to the opera- 
tional goals established by the institution. 

For a number of problems, then, we need a common 
language of pupil behavior. Even the transfer problem 
takes on a different complexion when, in addition to a Tank 
in class or a grade point, we can communicate five or 10 
critical skills which the student can now do as a result of 
having had English 201. At the present time we know only 
his grade in English 201, not what he can do, even after 
having read the catalog description. In some way, it is 
impossible to measure or communicate what we can't see. 
It is also difficult to plan without clear objectives. In my 
opinion, a first task is to specify educational objectives 
in operational terms. 



Charles Alexander, editor 
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